{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Debugging a pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`river` encourages users to make use of pipelines. The biggest pain point of pipelines is that it can be hard to understand what's happening to the data, especially when the pipeline is complex. Fortunately the `Pipeline` class has a `debug_one` method that can help out.\n",
    "\n",
    "Let's look at a fairly complex pipeline for predicting the number of bikes in 5 bike stations from the city of Toulouse. It doesn't matter if you understand the pipeline or not; the point of this notebook is to learn how to introspect a pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading https://maxhalford.github.io/files/datasets/toulouse_bikes.zip (1.12 MB)\n",
      "Uncompressing into /Users/max.halford/river_data/Bikes\n",
      "30000 MAE: 2.220942\n",
      "60000 MAE: 2.270271\n",
      "90000 MAE: 2.301302\n",
      "120000 MAE: 2.275876\n",
      "150000 MAE: 2.275224\n",
      "180000 MAE: 2.289347\n"
     ]
    }
   ],
   "source": [
    "import datetime as dt\n",
    "from river import compose\n",
    "from river import datasets\n",
    "from river import feature_extraction\n",
    "from river import linear_model\n",
    "from river import metrics\n",
    "from river import preprocessing\n",
    "from river import stats\n",
    "from river import stream\n",
    "\n",
    "\n",
    "X_y = datasets.Bikes()\n",
    "X_y = stream.simulate_qa(X_y, moment='moment', delay=dt.timedelta(minutes=30))\n",
    "\n",
    "def add_time_features(x):\n",
    "    return {\n",
    "        **x,\n",
    "        'hour': x['moment'].hour,\n",
    "        'day': x['moment'].weekday()\n",
    "    }\n",
    "\n",
    "model = add_time_features\n",
    "model |= (\n",
    "    compose.Select('clouds', 'humidity', 'pressure', 'temperature', 'wind') +\n",
    "    feature_extraction.TargetAgg(by=['station', 'hour'], how=stats.Mean()) +\n",
    "    feature_extraction.TargetAgg(by='station', how=stats.EWMean())\n",
    ")\n",
    "model |= preprocessing.StandardScaler()\n",
    "model |= linear_model.LinearRegression()\n",
    "\n",
    "metric = metrics.MAE()\n",
    "\n",
    "questions = {}\n",
    "\n",
    "for i, x, y in X_y:\n",
    "    # Question\n",
    "    is_question = y is None\n",
    "    if is_question:\n",
    "        y_pred = model.predict_one(x)\n",
    "        questions[i] = y_pred\n",
    "    \n",
    "    # Answer\n",
    "    else:\n",
    "        metric.update(y, questions[i])\n",
    "        model = model.learn_one(x, y)\n",
    "    \n",
    "        if i >= 30000 and i % 30000 == 0:\n",
    "            print(i, metric)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by looking at the pipeline. You can click each cell to display the current state for each step of the pipeline."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html><body><div class=\"pipeline\"><div class=\"estimator\"><pre>Features</pre></div><details class=\"estimator\"><summary><pre class=\"estimator-name\">add_time_features</pre></summary><code class=\"estimator-params\">\n",
       "def add_time_features(x):\n",
       "    return {\n",
       "        **x,\n",
       "        'hour': x['moment'].hour,\n",
       "        'day': x['moment'].weekday()\n",
       "    }\n",
       "\n",
       "</code></details><div class=\"union\"><details class=\"estimator\"><summary><pre class=\"estimator-name\">['clouds', 'humidity', 'pressure', 'temperature', 'wind']</pre></summary><code class=\"estimator-params\">\n",
       "{'whitelist': {'temperature', 'pressure', 'wind', 'humidity', 'clouds'}}\n",
       "\n",
       "</code></details><details class=\"estimator\"><summary><pre class=\"estimator-name\">target_mean_by_station_and_hour</pre></summary><code class=\"estimator-params\">\n",
       "{'by': ['station', 'hour'],\n",
       " 'feature_name': 'target_mean_by_station_and_hour',\n",
       " 'groups': defaultdict(functools.partial(&lt;function deepcopy at 0x7fb400fad790&gt;, Mean ()),\n",
       "                       {'metro-canal-du-midi_0': Mean (),\n",
       "                        'metro-canal-du-midi_1': Mean (),\n",
       "                        'metro-canal-du-midi_10': Mean (),\n",
       "                        'metro-canal-du-midi_11': Mean (),\n",
       "                        'metro-canal-du-midi_12': Mean (),\n",
       "                        'metro-canal-du-midi_13': Mean (),\n",
       "                        'metro-canal-du-midi_14': Mean (),\n",
       "                        'metro-canal-du-midi_15': Mean (),\n",
       "                        'metro-canal-du-midi_16': Mean (),\n",
       "                        'metro-canal-du-midi_17': Mean (),\n",
       "                        'metro-canal-du-midi_18': Mean (),\n",
       "                        'metro-canal-du-midi_19': Mean (),\n",
       "                        'metro-canal-du-midi_2': Mean (),\n",
       "                        'metro-canal-du-midi_20': Mean (),\n",
       "                        'metro-canal-du-midi_21': Mean (),\n",
       "                        'metro-canal-du-midi_22': Mean (),\n",
       "                        'metro-canal-du-midi_23': Mean (),\n",
       "                        'metro-canal-du-midi_3': Mean (),\n",
       "                        'metro-canal-du-midi_4': Mean (),\n",
       "                        'metro-canal-du-midi_5': Mean (),\n",
       "                        'metro-canal-du-midi_6': Mean (),\n",
       "                        'metro-canal-du-midi_7': Mean (),\n",
       "                        'metro-canal-du-midi_8': Mean (),\n",
       "                        'metro-canal-du-midi_9': Mean (),\n",
       "                        'place-des-carmes_0': Mean (),\n",
       "                        'place-des-carmes_1': Mean (),\n",
       "                        'place-des-carmes_10': Mean (),\n",
       "                        'place-des-carmes_11': Mean (),\n",
       "                        'place-des-carmes_12': Mean (),\n",
       "                        'place-des-carmes_13': Mean (),\n",
       "                        'place-des-carmes_14': Mean (),\n",
       "                        'place-des-carmes_15': Mean (),\n",
       "                        'place-des-carmes_16': Mean (),\n",
       "                        'place-des-carmes_17': Mean (),\n",
       "                        'place-des-carmes_18': Mean (),\n",
       "                        'place-des-carmes_19': Mean (),\n",
       "                        'place-des-carmes_2': Mean (),\n",
       "                        'place-des-carmes_20': Mean (),\n",
       "                        'place-des-carmes_21': Mean (),\n",
       "                        'place-des-carmes_22': Mean (),\n",
       "                        'place-des-carmes_23': Mean (),\n",
       "                        'place-des-carmes_3': Mean (),\n",
       "                        'place-des-carmes_4': Mean (),\n",
       "                        'place-des-carmes_5': Mean (),\n",
       "                        'place-des-carmes_6': Mean (),\n",
       "                        'place-des-carmes_7': Mean (),\n",
       "                        'place-des-carmes_8': Mean (),\n",
       "                        'place-des-carmes_9': Mean (),\n",
       "                        'place-esquirol_0': Mean (),\n",
       "                        'place-esquirol_1': Mean (),\n",
       "                        'place-esquirol_10': Mean (),\n",
       "                        'place-esquirol_11': Mean (),\n",
       "                        'place-esquirol_12': Mean (),\n",
       "                        'place-esquirol_13': Mean (),\n",
       "                        'place-esquirol_14': Mean (),\n",
       "                        'place-esquirol_15': Mean (),\n",
       "                        'place-esquirol_16': Mean (),\n",
       "                        'place-esquirol_17': Mean (),\n",
       "                        'place-esquirol_18': Mean (),\n",
       "                        'place-esquirol_19': Mean (),\n",
       "                        'place-esquirol_2': Mean (),\n",
       "                        'place-esquirol_20': Mean (),\n",
       "                        'place-esquirol_21': Mean (),\n",
       "                        'place-esquirol_22': Mean (),\n",
       "                        'place-esquirol_23': Mean (),\n",
       "                        'place-esquirol_3': Mean (),\n",
       "                        'place-esquirol_4': Mean (),\n",
       "                        'place-esquirol_5': Mean (),\n",
       "                        'place-esquirol_6': Mean (),\n",
       "                        'place-esquirol_7': Mean (),\n",
       "                        'place-esquirol_8': Mean (),\n",
       "                        'place-esquirol_9': Mean (),\n",
       "                        'place-jeanne-darc_0': Mean (),\n",
       "                        'place-jeanne-darc_1': Mean (),\n",
       "                        'place-jeanne-darc_10': Mean (),\n",
       "                        'place-jeanne-darc_11': Mean (),\n",
       "                        'place-jeanne-darc_12': Mean (),\n",
       "                        'place-jeanne-darc_13': Mean (),\n",
       "                        'place-jeanne-darc_14': Mean (),\n",
       "                        'place-jeanne-darc_15': Mean (),\n",
       "                        'place-jeanne-darc_16': Mean (),\n",
       "                        'place-jeanne-darc_17': Mean (),\n",
       "                        'place-jeanne-darc_18': Mean (),\n",
       "                        'place-jeanne-darc_19': Mean (),\n",
       "                        'place-jeanne-darc_2': Mean (),\n",
       "                        'place-jeanne-darc_20': Mean (),\n",
       "                        'place-jeanne-darc_21': Mean (),\n",
       "                        'place-jeanne-darc_22': Mean (),\n",
       "                        'place-jeanne-darc_23': Mean (),\n",
       "                        'place-jeanne-darc_3': Mean (),\n",
       "                        'place-jeanne-darc_4': Mean (),\n",
       "                        'place-jeanne-darc_5': Mean (),\n",
       "                        'place-jeanne-darc_6': Mean (),\n",
       "                        'place-jeanne-darc_7': Mean (),\n",
       "                        'place-jeanne-darc_8': Mean (),\n",
       "                        'place-jeanne-darc_9': Mean (),\n",
       "                        'pomme_0': Mean (),\n",
       "                        'pomme_1': Mean (),\n",
       "                        'pomme_10': Mean (),\n",
       "                        'pomme_11': Mean (),\n",
       "                        'pomme_12': Mean (),\n",
       "                        'pomme_13': Mean (),\n",
       "                        'pomme_14': Mean (),\n",
       "                        'pomme_15': Mean (),\n",
       "                        'pomme_16': Mean (),\n",
       "                        'pomme_17': Mean (),\n",
       "                        'pomme_18': Mean (),\n",
       "                        'pomme_19': Mean (),\n",
       "                        'pomme_2': Mean (),\n",
       "                        'pomme_20': Mean (),\n",
       "                        'pomme_21': Mean (),\n",
       "                        'pomme_22': Mean (),\n",
       "                        'pomme_23': Mean (),\n",
       "                        'pomme_3': Mean (),\n",
       "                        'pomme_4': Mean (),\n",
       "                        'pomme_5': Mean (),\n",
       "                        'pomme_6': Mean (),\n",
       "                        'pomme_7': Mean (),\n",
       "                        'pomme_8': Mean (),\n",
       "                        'pomme_9': Mean ()}),\n",
       " 'how': Mean (),\n",
       " 'target_name': 'target'}\n",
       "\n",
       "</code></details><details class=\"estimator\"><summary><pre class=\"estimator-name\">target_ewm_0.5_by_station</pre></summary><code class=\"estimator-params\">\n",
       "{'by': ['station'],\n",
       " 'feature_name': 'target_ewm_0.5_by_station',\n",
       " 'groups': defaultdict(functools.partial(&lt;function deepcopy at 0x7fb400fad790&gt;, EWMean (\n",
       "  alpha=0.5\n",
       ")),\n",
       "                       {'metro-canal-du-midi': EWMean (\n",
       "  alpha=0.5\n",
       "),\n",
       "                        'place-des-carmes': EWMean (\n",
       "  alpha=0.5\n",
       "),\n",
       "                        'place-esquirol': EWMean (\n",
       "  alpha=0.5\n",
       "),\n",
       "                        'place-jeanne-darc': EWMean (\n",
       "  alpha=0.5\n",
       "),\n",
       "                        'pomme': EWMean (\n",
       "  alpha=0.5\n",
       ")}),\n",
       " 'how': EWMean (\n",
       "  alpha=0.5\n",
       "),\n",
       " 'target_name': 'target'}\n",
       "\n",
       "</code></details></div><details class=\"estimator\"><summary><pre class=\"estimator-name\">StandardScaler</pre></summary><code class=\"estimator-params\">\n",
       "{'counts': Counter({'target_ewm_0.5_by_station': 182470,\n",
       "                    'target_mean_by_station_and_hour': 182470,\n",
       "                    'temperature': 182470,\n",
       "                    'pressure': 182470,\n",
       "                    'wind': 182470,\n",
       "                    'humidity': 182470,\n",
       "                    'clouds': 182470}),\n",
       " 'means': defaultdict(&lt;class 'float'&gt;,\n",
       "                      {'clouds': 30.315131254453505,\n",
       "                       'humidity': 62.24244533347998,\n",
       "                       'pressure': 1017.0563060996391,\n",
       "                       'target_ewm_0.5_by_station': 10.08331958752748,\n",
       "                       'target_mean_by_station_and_hour': 9.410348580619415,\n",
       "                       'temperature': 20.50980692716619,\n",
       "                       'wind': 3.4184331122924543}),\n",
       " 'vars': defaultdict(&lt;class 'float'&gt;,\n",
       "                     {'clouds': 1389.0025610928221,\n",
       "                      'humidity': 349.59967918503554,\n",
       "                      'pressure': 33.298307526514115,\n",
       "                      'target_ewm_0.5_by_station': 80.17355266024735,\n",
       "                      'target_mean_by_station_and_hour': 33.98249801051089,\n",
       "                      'temperature': 34.70701720774977,\n",
       "                      'wind': 4.473627075744674})}\n",
       "\n",
       "</code></details><details class=\"estimator\"><summary><pre class=\"estimator-name\">LinearRegression</pre></summary><code class=\"estimator-params\">\n",
       "{'_weights': {'target_ewm_0.5_by_station': 9.264175276315454, 'target_mean_by_station_and_hour': 0.19801400070497752, 'temperature': -0.4211217806219121, 'pressure': 0.181374989091379, 'wind': -0.04087954775180142, 'humidity': 1.012524843761298, 'clouds': -0.326969479445828},\n",
       " '_y_name': None,\n",
       " 'clip_gradient': 1000000000000.0,\n",
       " 'initializer': Zeros (),\n",
       " 'intercept': 9.223158690689168,\n",
       " 'intercept_init': 0.0,\n",
       " 'intercept_lr': Constant({'learning_rate': 0.01}),\n",
       " 'l2': 0.0,\n",
       " 'loss': Squared({}),\n",
       " 'optimizer': SGD({'lr': Constant({'learning_rate': 0.01}), 'n_iterations': 182470})}\n",
       "\n",
       "</code></details></div></body><style>\n",
       ".estimator {\n",
       "    padding: 1em;\n",
       "    border-style: solid;\n",
       "    background: white;\n",
       "}\n",
       "\n",
       ".pipeline {\n",
       "    display: flex;\n",
       "    flex-direction: column;\n",
       "    align-items: center;\n",
       "    background: linear-gradient(#000, #000) no-repeat center / 3px 100%;\n",
       "}\n",
       "\n",
       ".union {\n",
       "    display: flex;\n",
       "    flex-direction: row;\n",
       "    align-items: center;\n",
       "    justify-content: center;\n",
       "    padding: 1em;\n",
       "    border-style: solid;\n",
       "    background: white\n",
       "}\n",
       "\n",
       "/* Vertical spacing between steps */\n",
       "\n",
       ".estimator + .estimator,\n",
       ".estimator + .union,\n",
       ".union + .estimator {\n",
       "    margin-top: 2em;\n",
       "}\n",
       "\n",
       ".union > .estimator {\n",
       "    margin-top: 0;\n",
       "}\n",
       "\n",
       "/* Spacing within a union of estimators */\n",
       "\n",
       ".union >\n",
       ".estimator + .estimator,\n",
       ".pipeline + .estimator,\n",
       ".estimator + .pipeline,\n",
       ".pipeline + .pipeline {\n",
       "    margin-left: 1em;\n",
       "}\n",
       "\n",
       "/* Typography */\n",
       ".estimator-params {\n",
       "    display: block;\n",
       "    white-space: pre-wrap;\n",
       "    font-size: 120%;\n",
       "    margin-bottom: -1em;\n",
       "}\n",
       "\n",
       ".estimator > code {\n",
       "    background-color: white !important;\n",
       "}\n",
       "\n",
       ".estimator-name {\n",
       "    display: inline;\n",
       "    margin: 0;\n",
       "    font-size: 130%;\n",
       "}\n",
       "\n",
       "/* Toggle */\n",
       "\n",
       "summary {\n",
       "    display: flex;\n",
       "    align-items:center;\n",
       "    cursor: pointer;\n",
       "}\n",
       "\n",
       "summary > div {\n",
       "    width: 100%;\n",
       "}\n",
       "</style></html>"
      ],
      "text/plain": [
       "Pipeline (\n",
       "  FuncTransformer (\n",
       "    func=\"add_time_features\"\n",
       "  ),\n",
       "  TransformerUnion (\n",
       "    Select (\n",
       "      clouds\n",
       "      humidity\n",
       "      pressure\n",
       "      temperature\n",
       "      wind\n",
       "    ),\n",
       "    TargetAgg (\n",
       "      by=['station', 'hour']\n",
       "      how=Mean ()\n",
       "      target_name=\"target\"\n",
       "    ),\n",
       "    TargetAgg (\n",
       "      by=['station']\n",
       "      how=EWMean (\n",
       "        alpha=0.5\n",
       "      )\n",
       "      target_name=\"target\"\n",
       "    )\n",
       "  ),\n",
       "  StandardScaler (),\n",
       "  LinearRegression (\n",
       "    optimizer=SGD (\n",
       "      lr=Constant (\n",
       "        learning_rate=0.01\n",
       "      )\n",
       "    )\n",
       "    loss=Squared ()\n",
       "    l2=0.\n",
       "    intercept_init=0.\n",
       "    intercept_lr=Constant (\n",
       "      learning_rate=0.01\n",
       "    )\n",
       "    clip_gradient=1e+12\n",
       "    initializer=Zeros ()\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As mentioned above the `Pipeline` class has a `debug_one` method. You can use this at any point you want to visualize what happen to an input `x`. For example, let's see what happens to the last seen `x`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0. Input\n",
      "--------\n",
      "clouds: 88 (int)\n",
      "description: overcast clouds (str)\n",
      "humidity: 84 (int)\n",
      "moment: 2016-10-05 09:57:18 (datetime)\n",
      "pressure: 1,017.34000 (float)\n",
      "station: pomme (str)\n",
      "temperature: 17.45000 (float)\n",
      "wind: 1.95000 (float)\n",
      "\n",
      "1. add_time_features\n",
      "--------------------\n",
      "clouds: 88 (int)\n",
      "day: 2 (int)\n",
      "description: overcast clouds (str)\n",
      "hour: 9 (int)\n",
      "humidity: 84 (int)\n",
      "moment: 2016-10-05 09:57:18 (datetime)\n",
      "pressure: 1,017.34000 (float)\n",
      "station: pomme (str)\n",
      "temperature: 17.45000 (float)\n",
      "wind: 1.95000 (float)\n",
      "\n",
      "2. Transformer union\n",
      "--------------------\n",
      "    2.0 Select\n",
      "    ----------\n",
      "    clouds: 88 (int)\n",
      "    humidity: 84 (int)\n",
      "    pressure: 1,017.34000 (float)\n",
      "    temperature: 17.45000 (float)\n",
      "    wind: 1.95000 (float)\n",
      "\n",
      "    2.1 TargetAgg\n",
      "    -------------\n",
      "    target_mean_by_station_and_hour: 7.89396 (float)\n",
      "\n",
      "    2.2 TargetAgg1\n",
      "    --------------\n",
      "    target_ewm_0.5_by_station: 11.80372 (float)\n",
      "\n",
      "clouds: 88 (int)\n",
      "humidity: 84 (int)\n",
      "pressure: 1,017.34000 (float)\n",
      "target_ewm_0.5_by_station: 11.80372 (float)\n",
      "target_mean_by_station_and_hour: 7.89396 (float)\n",
      "temperature: 17.45000 (float)\n",
      "wind: 1.95000 (float)\n",
      "\n",
      "3. StandardScaler\n",
      "-----------------\n",
      "clouds: 1.54778 (float)\n",
      "humidity: 1.16366 (float)\n",
      "pressure: 0.04916 (float)\n",
      "target_ewm_0.5_by_station: 0.19214 (float)\n",
      "target_mean_by_station_and_hour: -0.26013 (float)\n",
      "temperature: -0.51938 (float)\n",
      "wind: -0.69426 (float)\n",
      "\n",
      "4. LinearRegression\n",
      "-------------------\n",
      "Name                              Value      Weight     Contribution  \n",
      "                      Intercept    1.00000    9.22316        9.22316  \n",
      "      target_ewm_0.5_by_station    0.19214    9.26418        1.78000  \n",
      "                       humidity    1.16366    1.01252        1.17823  \n",
      "                    temperature   -0.51938   -0.42112        0.21872  \n",
      "                           wind   -0.69426   -0.04088        0.02838  \n",
      "                       pressure    0.04916    0.18137        0.00892  \n",
      "target_mean_by_station_and_hour   -0.26013    0.19801       -0.05151  \n",
      "                         clouds    1.54778   -0.32697       -0.50608  \n",
      "\n",
      "Prediction: 11.87982\n"
     ]
    }
   ],
   "source": [
    "print(model.debug_one(x))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The pipeline does quite a few things, but using `debug_one` shows what happens step by step. This is really useful for checking that the pipeline is behaving as you're expecting it too. Remember that you can `debug_one` whenever you wish, be it before, during, or after training a model."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
